1. Field of the Invention
The present invention relates to an apparatus for performing computing operations according to the neural network theory. More particularly, the invention relates to a nonlinear operation unit of simple circuit constructions and improved accuracy, and a data processing apparatus using the nonlinear operation units.
2. Discussion of the Prior Art
A neural-network basis apparatus having the functions of computing operations, storage, learning and the like has been realized by circuits and operation means, which are modeled on each neuron. The fact that most of the outputs of nerves become the inputs of other nerves has been well known. Also in the neural-network basis apparatus, the output signal of the circuit and the operation means, which are modeled on a neuron, frequently become the input signal of the circuit and operation means, which are modeled on another neuron. By convention, a neural network is constructed by combining those neuron circuits and operation means. Accordingly, the neural network includes the neuron circuits and operation means, which are necessary for forming each neuron but are omissible in light of constructing the overall network. In other words, the conventional neural network must include unnecessary circuits and operation means.
The neuron linearly multiplies the signals from other neurons in accordance with a strength of bonding of synapses for transferring the signals, adds the results of multiplying operations from the synapses of the neuron to the threshold value of the neuron, and transfers the output signals, which are equivalent to the results of the process carried out by the nonlinear function, to other neurons, through the synapses. The conventional art constructs the neuron circuits and operation means in a similar order. An example of this is described in "High-Speed Analog Neural Network LSI for Neuro Processing Type Light Demultiplexes", Electric/Electronics/Information/Communication Academy NC90-53 (Article 1). The synapses of each neuron are constructed by a circuit corresponding to a linear multiplier circuit, invented by Gilbert and disclosed in Japanese Patent Laid-Open Publication No. Sho. 48-20932 (Article 2). The output currents of the synapses are added together. Then, a nonlinear characteristic is gained by a differential amplifier. The current is converted into a corresponding voltage, which is then used as an output signal of the neuron. A neural circuitry based on the neural circuitry theory is constructed by connecting the circuits modeled on the neurons. When carefully studying the Gilbert multiplier circuit, it is seen that the multiplier circuit is made up of a multiplier core circuit and a circuit for correcting the nonlinearity of the multiplier core circuit. These circuits each include a circuit for converting a voltage difference into a current difference. Generally, this voltage-to-current converting circuit suffers from an appreciable error. The circuit including diode-connected transistors also suffers from an appreciable error arising from a variation of the characteristics of the elements, which is caused during the manufacturing process. Therefore, it is impossible to perfectly correct the nonlinearity of the multiplier core circuit. The Gilbert multiplier circuit is attendant with an error, although appreciable. For this reason, an error owing to the nonlinearity is observed in the multiplier characteristic, as described on page 87, FIG. 5 in the article 2. The nonlinear function circuit using the differential amplifier, which is connected to the output of the multiplier circuit, also causes an error. The nonlinear function circuit is influenced by temperature of the elements. The input sensitivity of the circuit is proportional to absolute temperature. Consequently, the characteristic of the overall neural network also varies. When a signal passes through multiple stages of circuits, the error is accumulated to increase. In an extreme case, it is next to impossible to realize desired functions.
The computing operation of a nonlinear function is performed at the output of another neuron. If this computing operation is performed in the synapses, it little affects the operation of an overall neural network, when taking account of the fact that most of the output signal of a neuron become the input signal to another neuron.